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Description 

BACKGROUND OF THE INVENTION 

Field of the Invention 5 

[0001] The present invention relates to a communica- 
tion apparatus and method and to an image processing 
apparatus and method. 

Related Background Art 

[0002] A monitoring apparatus as an example of such 
communication apparatus is composed of several video 
cameras, a composer for performing analog synthesis 
of videos therefrom, and a device called a switcher for 
enabling selection thereof. Such apparatus are mainly 
used in relatively small buildings and are called local 
monitoring systems. In contrast with the local monitoring 
systems, recently being developed are remote monitor- 
ing systems which use a digital network such as LAN or 
ISDN instead of analog cables as video transmission 
lines, thereby permitting substantial extension of trans- 
mission lines. 

[0003] Camera control systems permitting remote 
control of one or plural monitoring cameras from a re- 
mote place are well known. 

[0004] Camera control systems permitting remote 
control of one or plural monitoring cameras from a re- 
mote place are well known. 

[0005] Monitoring cameras can be fixed so are always 
directed in one direction or can permit external conrol of 
pan, tilt, and zoom. The latters camera usually incorpo- 
rates a function to remotely control the taking direction 
and magnification from a monitoring center. 
[0006] When a plurality of cameras are remotely con- 
trolled, it is necessary to indicate locations and orienta- 
tions of the respective cameras for an operator. For ex- 
ample, proposed as a monitoring system with many 
cameras set in a shop, a plant, or a building arranged 
to remotely control them is a camera information display 
system for displaying graphics (camera icons) indicating 
the cameras at positions corresponding to the setting 
positions of the respective cameras. 
[0007] Among the recent monitoring systems there 
are some reported systems utilizing an IBM PC compat- 
ible (PC) as a monitoring terminal and realizing video 
display and system control with Graphical User Inter- 
face (GUI). Use of GUI with a computer as a monitoring 
apparatus presents an advantage of ease to handle 
even for people unaccustomed to the apparatus. 
[0008] However, the conventional systems of this type 
use the digital network as video transmission lines, but 
show analog display on the monitor. Thus, the display 
positions of video camera images strongly depend upon 
the hardware of system. Therefore, since the display po- 
sitions of video camera images are fixedly assigned to 
a video display section on a specific monitoring terminal 



upon introduction of system, an observer being a user 
cannot freely rearrange the video display position in or- 
der to make them easier to handle or cannot select the 
video camera images which are merely displayed. 
[0009] A concern of the present invention is to provide 
a communication apparatus and method easy to use, 
taking account of the foregoing points. 
[001 0] Another concern of the present invention is to 
realize easy line connection with a desired terminal. 
[0011] A further concern of the present invention is to 
realize easy-to-view observation of received images 
from image generating apparatus. 
[0012] European Patent Specification No EP-A- 
0513601 discloses a graphical work station for use with 
associated security device which displays in operation 
graphical images relevant to the operation of the secu- 
rity devices. 

[0013] In accordance with one aspect of the present 
invention there is provided a monitor control apparatus 
as set out in claim 1. 

[0014] In accordance with a second aspect there is 
provided a method of monitoring the operation of a plu- 
rality of remote cameras as set out in claim 10. 
[0015] Other advantages and features of the present 
invention will be apparent from the following description 
and the attached drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 
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30 [0016] 

Fig. 1 is a block diagram of video transmitting ter- 
minal and monitoring terminal in an embodiment of 
the present invention. 
35 Fig. 2 is a block diagram of software in an embodi- 
ment of the present invention. 
Fig. 3 is a screen example of the embodiment. 
Fig. 4 is a drawing to show a screen after switch of 
map to 530. 

40 Fig. 5 is a drawing to show an example of video dis- 
play window 600 of the embodiment. 
Fig. 6 is a drawing to show the state upon execution 
of display by D & D operation. 
Fig. 7 is a drawing to show an example of the shape 
45 of mouse cursor upon D & D operation. 

Fig. 8 is a drawing to show the state upon change 
of video display area by D & D operation. 
Fig. 9 is a drawing to show a display example of 
camera icon under video display. 
so Fig. 10 is a drawing to show the state upon execu- 
tion of video display stop by D & D operation. 
Fig. 11 is a flowchart to show the operation of the 
entire apparatus of the present embodiment. 
Fig. 12 is a flowchart to show the operation of the 
55 entire apparatus of the present embodiment. 

Fig. 13 is a flowchart to show the operation of the 
entire apparatus of the present embodiment. 
Fig. 14 is a flowchart to show the details of display 
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process routine in S110 shown in Fig. 11. 
Fig. 15 is a flowchart to show the details of display 
process routine in S110 shown in Fig. 11. 
Fig. 1 6 is a flowchart to show the details of start rou- 
tine of display movement and stop process in S118 5 
shown in Fig. 11. 

Fig. 1 7 is a flowchart to show the details of start rou- 
tine of display movement and stop process in S118 
shown in Fig. 11. 

Fig. 18 is a flowchart to show the configuration of 10 
camera control client shown in Fig. 2. 
Fig. 19 is a flowchart to show the details of display 
process routine in S1 10 shown in Fig. 11 , in Embod- 
iment 2. 

Fig. 20 is a flowchart to show the details of display t« 
process routine in S1 10 shown in Fig. 11 , in Embod- 
iment 2. 

Fig. 21 is a flowchart to show the details of start rou- 
tine of display movement and stop process in S118 
shown in Fig. 11, in Embodiment 2. 20 
Fig. 22 is a flowchart to show the details of start rou- 
tine of display movement and stop process in S118 
shown in Fig. 11, in Embodiment 2. 
Fig. 23 is a flowchart to show the details of start rou- 
tine of display movement and stop process in S118 25 
shown in Fig. 11, in Embodiment 2. 
Fig. 24 is a flowchart to show the details of display 
process routine in S1 10 shown in Fig. 11 , in Embod- 
iment 3. 

Fig. 25 is a flowchart to show the details of display 30 
process routine in S110 shown in Fig. 11, in Embod- 
iment 3. 

Fig. 26 is a flowchart to show the details of start rou- 
tine of display movement and stop process in S118 
shown in Fig. 11 , in Embodiment 3. 35 
Fig. 27 is a flowchart to show the details of start rou- 
tine of display movement and stop process in S118 
shown in Fig. 11, in Embodiment 3. 
Fig. 28 is a flowchart to show the details of start rou- 
tine of display movement and stop process in S118 *o 
shown in Fig. 11, in Embodiment 3. 
Fig. 29 is a block diagram to show a video commu- 
nication terminal apparatus (video transmission-re- 
ception apparatus), according to another embodi- 
ment of the present invention. 
Fig. 30 is a schematic view to illustrate a screen for 
displaying an image from a video transmission ap- 
paratus to which a plurality of cameras are connect- 
ed. 

Fig. 31 is a block diagram to show a video commu- so 
nication terminal apparatus in which a camera input 
composer is included. 

Fig. 32 is a schematic view to illustrate a screen for 
displaying an image from the video transmission 
apparatus in which the camera input composer is 55 
included and to which the plurality of cameras are 
connected. 

Fig. 33 is a schematic view to illustrate a screen for 



displaying a composite image from the video trans- 
mission apparatus in which the camera input com- 
poser is included and to which the plurality of cam- 
eras are connected. 

Fig. 34 is a block diagram to show the configuration 
of a network according to another embodiment of 
the present invention. 

Fig. 35 is a block diagram to show the configuration 
of software according to the embodiment of the 
present invention. 

Fig. 36 is a plan view to illustrate a camera display 
control panel which is displayed by a monitoring ap- 
paratus according to the present invention. 
Fig. 37 is a drawing to show a camera icon repre- 
senting a camera that cannot be controlled. 
Fig. 38 is a drawing to show a camera icon repre- 
senting a plurality of cameras connected to the 
same host. 

Fig. 39 is a drawing to show a camera icon repre- 
senting a camera connected to the camera input 
composer. 

Fig. 40 is a plan view to illustrate a camera display 
control panel, including a composite image, which 
panel is displayed by the monitoring apparatus. 
Fig. 41 is a partially enlarged view to show part of 
the composite image shown in Fig. 40. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

(Embodiment 1) 

[0017] Embodiment 1 of the present invention will be 
described in detail with reference to the drawings. 
[0018] Fig. 1 is a block diagram of the whole system 
including video transmitting terminals and a monitoring 
terminal. 

[0019] A video transmitting terminal 20 shown in Fig. 
1 is used together with an example of an image signal 
generating apparatus, in the present embodiment a vid- 
eo camera 60. Terminal 20 receives a camera control 
signal from a camera control terminal 12 to control the 
video camera, and also performs AID conversion and 
data compression of video signals from the video cam- 
era to transmit obtained data in this manner to the net- 
work. 

[0020] In Fig. 1, reference numeral 10 designates a 
video camera, and 1 2 a camera control circuit for directly 
controlling pan, tilt, zoom, focusing, aperture, and so on 
of the video camera 10 in accordance with an external 
control signal. The external control signal is input from 
an interface provided by RS-232C, for example. 
[0021] The camera control circuit 12 does not always 
have to be provided in the case of the video camera 10 
incapable of controlling pan, tilt, etc., but the control cir- 
cuit 1 2 may be arranged to perform on/off control of pow- 
er supply of video camera 10. Numeral 20 denotes a 
computer for controlling the video camera 10 connected 
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to the camera control circuit 12 by sending a control 
code to the camera control circuit 1 2 and for transmitting 
video data obtained from the video camera 10 through 
a network interface 38 to the network. For example, it is 
conceivably a work station (WS) or a personal computer 5 
(PC). 

[0022] Next described is the configuration of the com- 
puter 20 in the present embodiment. Numeral 22 repre- 
sents a CPU for controlling the entire system, 24 a main 
memory, 26 a secondary memory unit such as a hard 
disc unit, and 28 a mouse as a designating device, 
which, without having to be limited to the mouse, may 
be another device in the present invention, for example, 
a touch panel on a display 35. Numeral 30 is a keyboard. 
Numeral 25 denotes a removable external memory unit 
such as a floppy disc or a CDROM, and the software of 
this system may be read out of these media or the net- 
work interface 38 to be stored in the secondary memory 
unit 26. Numeral 32 is an I/O board connected to the 
camera control circuit 12 for transmitting and receiving 
the camera control signal, and 34 a video capture board 
for capturing a video output signal VD from the video 
camera 10. The video output signal may be either ana- 
log as in NTSC or digital, but in the analog case posses- 
sion of a function to perform AID conversion is required. 
The video capture board 34 does not have to include a 
data compression function, but without the compression 
function, execution of software compression is rather 
desired. The captured video is output from the board 34 
through a bus 39 to the video board 36 to be displayed 
at an arbitrary position on the bit map display 35. Such 
control of display position is carried out when the CPU 
22 gives an instruction of display position and position 
or area to the video board 36. Numeral 39 is a system 
bus for connecting the devices of from CPU 22 to net- 
work interface 38 with each other. 
[0023] The above configuration permits the terminal 
20 to transmit a video signal via the network 100 to the 
monitoring terminal at a remote place, to receive the 
camera control signal from the monitoring terminal, and 
to perform the camera control. Numeral 60 indicates the 
schematic configuration of the monitoring terminal (vid- 
eo receiving terminal). The monitoring terminal 60 
sends a control signal for video camera 10 to the video 
transmitting terminal 20, and the video transmitting ter- 
minal 20 controls the video camera in accordance with 
such a control signal and sends the status of video cam- 
era 10 obtained as a result back to the monitoring ter- 
minal 60. The monitoring terminal 60 displays the status 
of the pertinent video camera on a display unit, for ex- 
ample, on a bit map display 135. Further, the monitoring 
terminal 60 receives video data sent from the video 
transmitting terminal 20, expands the compressed data 
by software, i.e., decodes the coded data, and displays 
the thus decoded data on the display unit in real time. 
As seen from the drawing, the monitoring terminal 60 
has the same configuration as that attained by excluding 
the video camera 10, camera control unit 12, and cap- 



ture board 34 from the video transmitting terminal 20, 
and elements having the same functions as those in the 
terminal 20 are denoted by reference numerals with ad- 
dition of "100" to those in the terminal 20. Of course, 
they do not necessarily have to be excluded in particular. 
If expansion of compressed data takes too much time 
because of insufficient performance of CPU 122, ex- 
tended hardware with a compression-expanding func- 
tion may be added. 

[0024] The above configuration permits the terminal 
60 to receive the video from the video transmitting ter- 
minal 20 at a remote place via the network 100, to per- 
form expansion of compressed data, to display the video 
at an arbitrary place on the bit map display 135, i.e., on 
the monitor displaying unit, and to transmit a video cam- 
era control code according to a control instruction of vid- 
eo camera input through the keyboard 130 or mouse 
128 by the operator, to the video transmitting terminal. 
[0025] Fig. 2 is a drawing to show the software con- 
figuration of the present embodiment. Software 410 of 
the monitoring terminal 60 and software 420 of plural 
video communication terminals 20 and the like is cpn- 
nected to the network 100, and installed in the software 

410 of the monitoring terminal is camera control client 

411 for remote-controlling cameras each connected to 
the software 420 of video communication terminals on 
the network, video reception software 41 2 for expanding 
compressed video data sent in the form of packet from 
the video communication terminal and displaying the ex- 
panded data, and map management software 413 hav- 
ing a GUI capable of graphically displaying a map, a 
camera symbol, positions of cameras by scope display 
of Fig. 6 as detailed hereinafter, and pan and zoom and 
performing camera control. The video reception soft- 
ware 412 is software playing a main role for performing 
management of cameras of the all video transmitting ter- 
minals connected to the network 100, which retains 
camera names of the respective cameras, host names 
of the computers to which the cameras are connected, 
information about the cameras including the camera 
status of pan/tilt, zoom, and the like and about whether 
they are controllable or not, and information indicating 
the present status about which camera is under control 
at present and a video of which camera is under display. 
The camera control client 411 and map management 
software 413 also shares these information to utilize it 
fordisplay change of camera symbol or the like. Numer- 
al 421 is software for controlling the conditions of cam- 
era including pan/tilt,. zoom, white balance, and so on 
for the camera connected to such a terminal and for stor- 
ing the video data output from the camera, which is 
called a camera control server. Numeral 422 designates 
software working together with the video reception soft- 
ware. 

[0026] Fig. 3 shows an example of the screen dis- 
played on the bit map display 1 35 of the monitoring ter- 
minal 410. Numeral 500 represents a map window, in 
which a plurality of maps 520, 530, 540,..., each indicat- 
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ing a layout of an office, a shop, or a warehouse, are 
managed in the present embodiment. There is no spe- 
cific limitations on the number of maps, which can be 
determined depending upon the performance of sys- 
tem. A tag is attached to each map, and by clicking this 
tag with a mouse pointer, a map with the pertinent tag 
is displayed on a map display area 502. At the same as 
it, camera symbols are also displayed as being posi- 
tioned on the pertinent map. Fig. 3 shows a state in 
which the map 520 among the maps 510 to 540 is dis- 
played on the map display area 502 and in which cam- 
era icons 521, 522, 523,... located on the map 520 are 
displayed. Here, with clicking the tag of map 530, the 
map 530 is displayed on the map display area 502 as 
shown in Fig. 4, and at the same time, camera icons 
531, 532, 533,... located on the map 530 are displayed. 
Fig. 5 is a drawing to show a window 600 to display input 
video signals, and this Fig. 5 shows an example in which 
the video signals received through the network 100 are 
displayed in respective windows. 
[0027] In Fig. 5 numeral 600 designates a video dis- 
play window, and areas 610, 612, 614, 616, 618, 620,... 
in the video display window are video display areas. 
There are six video display areas in the present embod- 
iment, but the number of areas is not limited to this, of 
course. Further, the map window 500 shown in Fig. 3 
and the video display window 600 shown in Fig. 5 may 
be displayed on a same screen or may be displayed on 
separate screens, i.e., on separate monitor units. Nu- 
meral 632 represents a garbage can icon for deleting a 
displayed camera video from a video display area. Nu- 
meral 640 denotes a panel for camera control, which 
has a variety of buttons for camera control and which 
can control pan/tilt and zoom of a selected camera. The 
panel 640 may be also displayed on the same screen 
as the windows 500, 600, or may be displayed on a dif- 
ferent screen therefrom. Since how to display the 
screens departs from the scope of the present invention, 
the details thereof are omitted herein. 
[0028] The graphical user interface GUI of the moni- 
toring system of the present embodiment will be next 
described using Fig. 6 to Fig. 10. The present embodi- 
ment is arranged so that when the drag and drop oper- 
ation (hereinafter referred to as D & D) is performed to 
drag and drop a camera icon on the map 520, 530,... to 
an arbitrary video display area in the video display win- 
dow, a moving picture from a camera corresponding to 
the icon dragged and dropped is displayed on the video 
display area where the icon is dropped. Fig. 6 shows a 
state upon D & D of video camera 523 to video display 
area 614. During drag the shape of the mouse cursor 
turns to the shape of camera as shown in Fig. 7, where- 
by the user can recognize that the operation of drag and 
drop is under way. At this time the map management 
software 413 searches the position information of the 
dragged camera for an ID number of the camera and 
sends the ID of the camera subjected to D & D to the 
video reception software 412, and the video reception 



software checks pan/tilt of camera, camera name, and 
host name to which the camera is connected, from this 
ID and sends such information to the camera control cli- 
ent 411 and to the map management software 413. 

s Based on this information, the camera control client 41 1 
starts network communication with the camera control 
server of the video transmitting terminal to which the 
pertinent camera is connected. After that, camera con- 
trol is performed between them, and information about 

10 pan/tilt or the like of the camera is always sent from the 
camera control client 41 1 to the video reception software 
412. The map management software performs updating 
of the maps stored in the video board 136 so as to 
change orientations of camera icons in correspondence 

15 to orientations of the actual cameras, to draw a scope 
910 indicating progress of display as shown in Fig. 9, 
and to draw a control pointer 920 for controlling pan/tilt, 
and zoom of camera, in the scope 910. 
[0029] This video camera display function is arranged 

20 so that when a video of another video camera is already 
displayed in a video display area of the video window 
for display, the previous display video is stopped and 
the new video is displayed, thus preforming replace- 
ment of display video. 

25 [0030] The map management software is always no- 
tified of the information about pan/tilt of camera, etc. 
from the video reception software 41 3, so that whenever 
the conditions of pan/tilt, zoom, etc. of camera are 
changed on the camera control panel 640, the change 

30 js reflected immediately to the camera icons 521 , 522, 
523,..., 531 , 532, 533,... Transmission of actual video is 
carried out based on a request from the video reception 
software 412. The video reception software 412 re- 
quests the video transmitting software 422 of the video 

35 transmitting terminal to which the pertinent camera is 
connected, to send data of one frame through the net- 
work 100. Receiving this request, the video transmitting 
software 422 divides data of the latest frame captured 
into packets and sends them to the video reception soft- 

40 ware 412. The video reception software 412 recon- 
structs the frame from the packets to display it in the 
pertinent video display area and again requests trans- 
mission of video. Repeating this at high speed, videos 
at a remote place are transmitted and displayed through 

45 the network. This permits the moving picture of video 
camera to be displayed on the bit map display. Display 
of videos of plural cameras can be realized by repeti- 
tively performing the processes of issue of video trans- 
mission request, compression of captured video, packet 

50 division, network transmission, packet reception, frame 
reconstruction, expansion of compressed data, and dis- 
play in order against the video transmission softwares 
to which the respective cameras are connected. 
[0031] Movement of the display position of video cam- 

55 era image can be realized by performing the D & D op- 
eration of the displayed video to a video display area 
desired as a destination, as shown in Fig. 8. Fig. 8 shows 
a state in which the video of video camera 523 having 
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been displayed in the video display area 614 was moved 
to 612. This moving function is arranged so that when 
another video camera image is already displayed in the 
destination video display area, the previous display vid- 
eo is stopped and the new video is displayed therein, 5 
thus performing replacement of display video. 
[0032] At this time the video reception software 412 
clears the pertinent video display area and changes in- 
ternal parameters so as to designate the D & D destina- 
tion as an area for the video of the pertinent video cam- 10 
era to be displayed. Thereafter, the video is displayed 
in the D & D destination. This operation will never inter- 
rupt the logical network connection. Namely, the net- 
work once connected is maintained without being inter- 
rupted until the video display area is dragged and is 
dropped to the garbage can icon 632 as detailed here- 
inafter. For stopping the display of video camera image, 
the video display area in which the video of video cam- 
era desired to stop displaying is displayed is dragged 
and dropped to the garbage can icon 632 in the video 20 
display window, as shown in Fig. 10, whereby display of 
the pertinent video can be stopped. Fig. 10 shows a 
state after execution of display stop of the video of video 
camera 523 having been displayed in the video display 
area 614. At this time the video reception software 412 25 
clears the pertinent video display area and stops issue 
of video transmission request to the pertinent video 
transmission software. Further, it notifies the camera 
control client 411 and map management software 413 
of stop of display. Receiving this notification, the camera 30 
control client 411 interrupts the network connection with 
the video transmission terminal and clears the pertinent 
video display area. Further, the map management soft- 
ware 413 removes the scope display from the camera 
icon 523 of the pertinent camera to update the map. 35 
[0033] The software of the apparatus in the present 
embodiment as described above will be explained re- 
ferring to Fig. 11. 

[0034] Fig. 11 to Fig. 13 are a flowchart to show the 
operation of the whole apparatus of the present embod- *o 
iment. 

[0035] Steps in the flowchart will be explained. 
[0036] S100: The map management software 413 
shown in Fig. 2 displays the map window, for example 
denoted by 500 in Fig. 3, on the bit map display 35. 45 
[0037] S102: After S100, the video reception software 
412 shown in Fig. 2 displays the video window 600, for 
example shown in Fig. 5, on the same display 35. 
[0038] S104: After S102, the camera control client 
411 shown in Fig. 2 displays the camera control panel 50 
640 shown in Fig. 6 on the same display 35. 
[0039] S106: It is determined whether a camera icon 
displayed on the map window shown in Fig. 3 is clicked; 
if clicked then the flow goes to S108; if not clicked then 
the flow goes to S1 12. 55 
[0040] Here, though the sequence of description is in- 
verted, it is noted that the steps of Fig. 11 are carried 
out by OS compatible with the CPU 122. In the above 



description S100, S102, and S104 are executed sepa- 
rately, but the step of S100 may be arranged to invoke 
the routines of S102 and S104 to carry out them. 
[0041] S108: Switching of focus clicked is executed. 
Namely, the clicked camera is fixed. In other words, it is 
determined which camera is the selected camera. 
[0042] S110: A display process routine detailed here- 
inafter is started. Such a routine will be described here- 
inafter with reference to Fig. 14. 
[0043] S112: It is determined whether a desired video 
out of the videos displayed on the video window 600 
shown in Fig. 5 is clicked. If clicked then the flow goes 
to S1 14, If not clicked then the flow proceeds to S120. 
[0044] S114: When a video is clicked, it is determined 
whether the position of a camera generating a signal of 
the clicked video is displayed on the map window shown 
in Fig. 3. If not displayed, the map management soft- 
ware 413 is instructed with an instruction for executing 
such switching of map as to display a map including the 
camera in the window shown in Fig. 3. 
[0045] S116: Executed is switching of focus of the 
camera corresponding to the clicked video in the same 
manner as in S1 08. 

[0046] S118: A display movement control and stop 
process routine detailed hereinafter is started. 
[0047] Such a routine will be described hereinafter 
with reference to Fig. 16. 

[0048] S120: It is determined whether the camera 
control panel 640 shown in Fig. 6 is clicked. When 
clicked, the flow branches to S122. 
[0049] S122: A command designated by the control 
panel is transmitted through the network 100 to the 
clicked camera. 

[0050] S124: It is determined whether either one of 
the tags 51 0 to 540 of the map shown in Fig. 3 is clicked. 
When clicked, the flow proceeds to S126. If not clicked, 
the flow proceeds to S128. 

[0051] S126: The map management software 413 is 
so instructed as to display a map corresponding to the 
clicked map out of the tags 510 to 540 among the maps. 
[0052] S1 28: It is determined whether there is a video 
designated as a video to be displayed by D & D on the 
map displayed on the bit map display 35. If there is a 
designated video, the flow goes to S130; if not the flow 
goes to S 136. 

[0053] S130: If there is a designated video, a packet 
including a video request signal is transmitted sequen- 
tially to a camera or to a video signal transmitting termi- 
nal corresponding to the video, and a video transmitted 
from the terminal requested by the packet is displayed 
on the bit map display 35. Such a packet is provided 
with an address on the network for designating the cam- 
era or video signal transmitting terminal corresponding 
to the video. 

[0054] S1 32: It is determined whether a grouping but- 
ton is clicked. If clicked the flow proceeds to S134. If not 
clicked the flow proceeds to C. 
[0055] S1 34: Cameras transmitting their camera vid- 
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eos on the bit map display 35 are grouped and a group 
icon is formed. 

[0056] S1 36: It is determined whether the group icon 
is clicked. If the group icon is clicked then the flow goes 
to S1 38. If not the flow goes to B. 5 
[0057] S138: The clicked group icon is displayed in 
reverse video. 

[0058] S140: It is determined whether a display in- 
struction of the group icon is given. If the display instruc- 
tion is given the flow goes to a group display flow of 10 
S142. Since the group display flow is not embraced in 
the essence of the present invention, the description 
thereof is omitted herein. If the display instruction is not 
given then the flow goes to S144. 

[0059] S144: It is determined whether a group dele- *5 
tion button 710 is clicked. When the group deletion but- 
ton 710 is clicked, the flow goes to S146. When the 
group deletion button 710 is not clicked, the flow goes 
toB. 

[0060] S146: The designated group is deleted. 20 
[0061] The display process routine described in S110 
will be next explained referring to Fig. 14. 
[0062] S150: In such a routine, first, it is determined 
whether a camera icon on display is dragged. In such a 
flow, if the user continues clicking the camera icon from 25 
S106 of Fig. 11 to this step, the flow goes to S1 51. If the 
user releases clicking, this step is ended and the flow 
returns to the original routine. 

[0063] S151: The mouse cursor is changed to such 
an indication as shown in Fig. 7 during dragging in order 30 
to permit the user to easily recognize that the camera is 
being dragged. 

[0064] The mouse cursor does not have to be limited 
to this example, but may be displayed in another display 
form. The point is that the user can recognize that the 35 
camera is under drag. 

[0065] S1 52: Whether the camera of the cursor shape 
exemplified in Fig. 7 is dropped is determined from the 
status of the click button of the mouse 128. 
[0066] It is determined whether the dropped camera *o 
is dropped into either one of the video display areas 610 
to 620 shown in Fig. 5. This determination is executed 
in such a way that the map management software 413 
compares coordinates between each area of the video 
display areas 61 0 to 620 and a position where the cursor 
of the camera shape was dropped; if the cursor is 
dropped in an area then the flow branches to S154; if 
the cursor is dropped outside of the areas then the flow 
branches to S168 to put the mouse cursor back. Name- 
ly, the cursor is returned to the status before the camera 50 
is dragged in S150. 

[0067] S154: The map management software 41 3 de- 
termines whether the video of the dropped area is being 
displayed. 

[0068] The map management software 41 3 performs 55 
such management as to always write in a table memory, 
information as to which video display area 610 to 620 
as described previously is displaying a video from which 



camera. 

[0069] If the dropped area is being displayed then the 
flow branches to S1 56; If not the flow branches to S1 58. 
[0070] S156: If a video is already being displayed in 
the dropped area, display of the video under display is 
stopped (which is equivalent to dropping it into the gar- 
bage can). 

[0071] S158: The video reception software 412 of Fig. 
2 is informed of ID (identification information) of the cam- 
era subjected to D & D. 

[0072] S159: Next obtained from the main memory 
unit 124 are the network address of the camera the ID 
of which was informed in S158, host name, camera 
name, and parameters indicating the status of the cam- 
era. Such parameters include focal length information, 
and data about a direction of white balance photogra- 
phy. The main memory unit 124 stores the network ad- 
dresses, host names, camera names, and parameters 
indicating the camera status of the all image signal gen- 
erators composing this system. 
[0073] S160: The data obtained in S159 is output to 
the map management software 413 and to the camera 
control client 411. 

[0074] S1 62: Based on the data acquired in S1 60, the 
scope display of the camera, for example, indicated by : 
580 in Fig. 6, which includes the angle of the field taken 
by the camera and the direction in which the camera of 
the focal length information is directed is displayed on 
the map. 

[0075] S164: After S162, communication with the 
camera control server 421 is started. 
[0076] S166: Then the video reception software 412 
is instructed so as to transmit a video request signal for 
receiving a video signal from the selected camera, to 
the selected camera. The video reception software 412 
repetitively outputs such video request signals regularly 
after it sends the video request signal to the selected 
camera and before transmission of video signal is 
stopped in S178 described hereinafter. 
[0077] This causes the video camera to sequentially 
output image signals of each frame repetitively, whereby 
a moving picture is reproduced on the monitor. 
[0078] S168: The mouse cursor having been moved 
by D & D is returned to the original position. 
[0079] The display movement and stop process rou- 
tine shown in S118 of Fig. 11 is next described with ref- 
erence to Fig. 16 and Fig. 17. 
[0080] S170: It is determined whether the video 
clicked in S112 is continuously being clicked. If contin- 
uously being clicked the flow goes to S172; if clicking is 
already released, this routine is ended. 
[0081] S172: The mouse cursor is changed to the 
shape, for example as shown in Fig. 7, in the same man- 
ner as in S1 51. 

[0082] S174: It is determined whether the video 
dragged by the mouse cursor is dropped into the gar- 
bage can. If dropped into the garbage can the flow goes 
to S 176; if not the flow goes to S 182. 
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[0083] S1 76: If the video is dropped into the garbage 
can, the video display area subjected to D & D is cleared 
to display nothing. Alternatively, the display is changed 
to blue back. 

[0084] S178: Issue of video transmission request is 5 
stopped for the transmission terminal, forexample a vid- 
eo camera, having been transmitting the video cleared 
in S 1 76 to the video reception software 41 2. This causes 
the communication terminal to stop communication of 
video signal. 10 
[0085] S180: After S178, the map management soft- 
ware 413'and camera control client 411 is informed of 
stop of display on the bit map display 135, and then the 
flow moves toS192. 

[0086] S182: If the camera video is not dropped into 1$ 
the garbage can icon in S1 74, it is determined into which 
video display area 610 to 620 the camera video was 
dropped. 

[0087] S184: The same as S154 
[0088] S186:ThesameasS156 . 20 

[0089] S188: The video signal of the camera under D 
& D is displayed at a video display position where the 
icon was dropped. 

[0090] S190: A video signal in the area in which the 
video image of camera has been displayed heretofore 25 
is cleared. 

[0091] S192: The scope display of camera displayed 
in S162 is erased. 

[0092] S194: Communication with the camera server 
is stopped. 30 
[0093] S196:ThesameasS168 
[0094] The present embodiment comprises, as de- 
scribed above, the means for dragging and dropping the 
camera symbol on the map to a video display area, 
thereby establishing the logical network connection be- 35 
tween the monitoring terminal and the pertinent video 
transmission terminal, the means for moving a video dis- 
play area displaying a video camera image to another 
video display area by D & D, and the means for dragging 
and dropping a video display area displaying a video *o 
camera image to the display stop symbol, thereby dis- 
connecting the logical network communication, which 
permits an observer as a user to freely rearrange the 
display positions of video camera images so as to be 
easy to handle, and to select video camera images to 45 
be displayed simultaneously, thereby giving flexibility to 
operation of the monitoring system, 

(Embodiment 2) 

50 

[0095] Embodiment 2 has the same basic processes 
as Embodiment 1, but is characterized in that upon the 
D & D operation of the video camera on the map to a 
video display area in order to display a video of the de- 
sired video camera in the video display area of the video 55 
window as shown by the operation in Fig. 6, if another 
video camera image is already displayed in that video 
display area, that event is detected and the display op- 



eration of that video is stopped. A flowchart to show the 
operation of software of this part is shown in Fig. 19 and 
Fig. 20. 

[0096] Further, the present embodiment also has 
such a feature that upon the D & D operation to a video 
display area expected as a destination in order to move 
the display position of the video camera image as shown 
by the operation in Fig. 8, if another video camera image 
is already displayed in that video display area, that event 
is detected and the movement operation of that video is 
stopped. A flowchart to show the operation of software 
of this part is shown in Figs. 21, 22, and 23. 
[0097] The steps in the flowcharts shown in Fig. 19 to 
Fig. 23 will be described. 
[0098] S198: The same as S150 
[0099] S200: The same as S1 51 
[01 00] S202: The same as S1 52 
[0101] S204: The map management software 41 3 de- 
termines whether a video is being displayed in the 
dropped area. The map management software 41 3 per- 
forms such management as to always write in the table 
memory the information as to a video of which camera 
is being displayed in which video display area 610 to 
620 shown in Fig. 5. If a video is not being displayed 
then the flow goes to S206; if not being displayed then, 
the flow branches to S218. 



[0102] 


S206: 


The 


same 


as 


S158 


[0103] 


S208: 


The 


same 


as 


S159 


[0104] 


S210: 


The 


same 


as 


S160 


[0105] 


S212: 


The 


same 


as 


S162 


[0106] 


S214: 


The 


same 


as 


S164 


[0107] 


S216: 


The 


same 


as 


S166 


[0108] 


S218: 


The 


same 


as 


S168 


[0109] 


S220: 


The 


same 


as 


S170 


[0110] 


S222: 


The 


same 


as 


S172 


[0111] 


S224: 


The 


same 


as 


S174 


[0112] 


S226: 


The 


same 


as 


S176 


[0113] 


S228: 


The 


same 


as 


S178 


[0114] 


S230: 


The 


same 


as 


S180 


[0115] 


S232: 


The 


same 


as 


S182 



[01 1 6] S234: The map management software 41 3 de- 
termines whether a video is being displayed in the 
dropped area. The map management software 413 per- 
forms such management as to always write in the table 
memory, the information as to a video of which camera 
is being displayed in which video display area 610 to 
620 shown in Fig. 5. If a video is not being displayed the 
flow goes to 5236. If a video is being displayed the flow 
branches to 5244. 
[0117] S236: The same as S188 
[01 1 8] S238: The same as S1 90 
[0119] S240: The same as S192 
[0120] S242: The same as S194 
[0121] S244: The same as S196 
[0122] The present embodiment is arranged so that 
upon the D & D operation of the camera symbol on the 
map to a video display area and upon the D & D oper- 
ation to move a video display area displaying a video 
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camera image to another arbitrary video display area, if 
another video camera image is already displayed in the 
pertinent area, such video display operation is stopped, 
whereby the video camera image already displayed can 
be prevented from being erased unintentionally. 5 

(Embodiment 3) 

[0123] Embodiment 3 has the same basic processes 
as Embodiment 1 , but has a function to switch the op- 
eration in accordance with binary information prelimi- 
narily stored, upon the D & D operation of a video cam- 
era on the map to a video display area in order to display 
a video of the desired video camera in a video display 
area of the video window, as shown by the operation in 
Fig. 6. Namely, the present embodiment is provided with 
two display modes, which are a display mode to give a 
higher priority to a video display already displayed 
(called a previous video) and a display mode to give a 
higher priority to a video display displayed newly (called 
a subsequent video), and it is arranged to preliminarily 
hold information about selection thereof and to keep the 
display of the previous video but stop display movement 
of the subsequent video in the case of the former display 
mode. In the case of the latter display mode, the subse- 
quent video is displayed in a desired video display area 
and the display of the previous video is stopped, thus 
effecting replacement of video. A flowchart to show the 
operation of software of this part is shown in Figs. 24 
and 25. 

[0124] Further, the present embodiment also has a 
function to switch the operation in accordance with bi- 
nary information preliminarily stored, upon the D & D op- 
eration to a video display area expected as a destination 
in order to move the display position of video camera 
image as shown by the operation in Fig. 8. Namely, the 
present embodiment is provided with two modes, which 
are a display mode to give a higher priority to a video 
display already displayed (called a previous video) and 
a display mode to give a higher priority to a video display 
displayed newly (called a subsequent video), and it is 
arranged to preliminarily hold information to select one 
of them and to keep the display of the previous video as 
it is, but stop display movement of the subsequent video, 
in the case of the former display mode. Further, in the 
case of the latter display mode, the subsequent video is 
moved to a desired video display area, and display of 
the previous video is stopped, thus effecting replace- 
ment of video. A flowchart to show the operation of soft- 
ware of this part is shown in Figs. 26, 27, and 28. 
[0125] The steps in the flowcharts shown in Fig. 24 to 
Fig. 28 will be described below. 
[0126] S246: The same as S150 
[0127] S248: The same as S151 
[0128] S250:ThesameasS152 
[0129] S252: The map management software 41 3 de- 
termines whether a video is already displayed in the ar- 
ea after D & D. The map management software 41 3 per- 



forms such management as to always write in the table 
memory the information as to a video of which camera 
is displayed in which video display area 610 to 620 
shown in Fig. 5. If a video is not displayed the flow 
branches to S258. If a video is displayed the flow goes 
to S254. 

[0130] S254: The map management software 413 
holds information about the display mode to display the 
video camera image in the video display area. This in- 
formation is binary information for discriminating a mode 
for stopping a subsequent video display with giving a 
higher priority to display of previous video (called a pre- 
vious screen priority mode) from a mode for replacing 
the display image with giving a higher priority to display 
of subsequent video (called a subsequent screen prior- 
ity mode) when a new camera video image is attempted 
to be displayed in a video display area and if another 
camera video image is already displayed therein, and a 
value of the information is preliminarily stored. 
[0131] The processing herein, depending upon the 
display mode, branches to S270 in the previous screen 
priority mode or proceeds to S256 in the subsequent 
screen priority mode. 



[0132] 


S256: 


The 


same 


as 


S156 


[0133] 


S258: 


The 


same 


as 


S158 


[0134] 


S260: 


The 


same 


as 


S162 


[0135] 


S262: 


The 


same 


as 


S164 


[0136] 


S264: 


The 


same 


as 


S159 


[0137] 


S266: 


The 


same 


as 


S160 


[0138] 


S268: 


The 


same 


as 


S166 


[0139] 


S270: 


The 


same 


as 


S168 


[0140] 


S272: 


The 


same 


as 


S170 


[0141] 


S274: 


The 


same 


as 


S172 


[0142] 


S276: 


The 


same 


as 


S174 


[0143] 


S278: 


The 


same 


as 


S176 


[0144] 


S280: 


The 


same 


as 


S178 


[0145] 


S282: 


The 


same 


as 


S180 


[0146] 


S284: 


The 


same 


as 


S182 



[0147] S286: The map management software 41 3 de- 
termines whether a video is already displayed in the ar- 
ea after D & D. The map management software 41 3 per- 
forms such management as to always write in the table 
memory the information as to a video of which camera 
is being displayed in which video display area 610 to 
620 shown in Fig. 5. If a video is not displayed the flow 
goes to S292. If a video is displayed the flow goes to 
S288. 

[0148] S288: The map management software 413 
holds the display mode information about the display 
mode to display the video camera image in the video 
display area. This information is binary information to 
give a criterion for selection between the mode to stop 
movement of the subsequent video with giving a higher 
priority to display of previous image (called a previous 
screen priority mode) and the mode to replace the video 
image with giving a higher priority to display of subse- 
quent video (called a subsequent screen priority mode) 
when the camera video image is moved to be displayed 



15 



20 



25 



30 



35 



40 



45 



50 



17 



EP 0 781 049 B1 



18 



in a video display area and when another camera video 
image is already displayed therein, and a value thereof 
is preliminarily stored. 

[0149] The processing herein, depending upon this 

display mode, branches to S300 in the previous screen 5 

priority mode or proceeds to S290 in the subsequent 

screen priority mode. 

[01 50] S290: The same as S1 86 

[0151] S292: The same as S188 

[0152] S294:ThesameasS190 10 
[01 53] S296: The same as S1 92 
[0154] S298: The same as S194 
[0155] S300: The same as S196 
[01 56] The present embodiment comprises the treat- 
ment method for the cases wherein upon new video dis- *5 
play by the D & D operation of the camera symbol on 
the map to a desired video display area and upon the D 
& D operation to move a video display area displaying 
a video camera image to another desired video display 
area, another camera video is already displayed in the 20 
pertinent video area, which is arranged to enable to se- 
lect either one of the following two display methods, ac- 
cording to the display mode information preliminarily 
stored. Namely, the present embodiment permits one to 
select either of the two display methods, which are the 25 
display method to give a higher priority to a camera vid- 
eo having been displayed heretofore and thereafter to 
stop display of a camera video intended to be newly dis- 
played or to be moved, and the display mode to give a 
higher priority to the camera video intended to be newly 30 
displayed or to be moved and to stop display of the cam- 
era video having been displayed heretofore, thus re- 
placing the video display screen of the pertinent display 
area. 

[0157] Since the embodiments are arranged to per- 35 
form network connection based on a display instruction 
to a specific position, in other words, since the network 
connection is not carried out based on a display instruc- 
tion to a position other than the specific position, erro- 
neous network connection can be prevented. *o 
[0158] By referring to the attached drawings, other 
embodiments of the present invention will be described 
below. 

[01 59] In accordance with another embodiment of the 
present invention, a plurality of video communication 45 
terminals (substantially computers having the function 
of video communication) which connect with one cam- 
era device or a plurality of camera devices are connect- 
ed to a computer network. Any of the communication 
terminals or an exclusive computer displays an image so 
from a selected camera device or images from selected 
camera devices, and operates each camera device by 
remote control. Fig. 29 shows the schematic block dia- 
gram of a typical example of the plurality of video com- 
munication terminals which connect with the one cam- 55 
era device or a plurality of camera devices. 
[0160] In Fig. 29, camera control circuits 12-1 to 12-3 
directly control video cameras 10-1 to 10-3 to pan, tilt, 



zoom, adjust focuses, adjust exposure, and perform 
other operations, in accordance with external control 
signals. A camera input selector 14 selects which of the 
video cameras 10-1 to 10-3 to control, in order to take 
in an output signal from the selected camera (the output 
signal is normally a video signal, but both a video signal 
and an audio signal when a microphone-equipped cam- 
era is used). As a communications line, for example, 
RS-232C and so forth are used, however, it is obvious 
that the present invention is not limited thereto, 
[0161] A video communication terminal apparatus 
(video transmitter-receiver apparatus) 20 sends a con- 
trol command to a desired camera controller 12-1, 12-2 
or 12-3 through the camera input selector 14 to control 
one of the video cameras 10-1 to 10-3 connected to the 
selected camera controller. This video communication 
terminal apparatus 20 also transmits a video signal from 
the selected camera to a network, while it receives a 
video signal from the network. The video communica- 
tion terminal apparatus 20 includes a central processing 
unit (CPU) 22 for controlling the whole of the apparatus 
20, a main memory 24, a second memory unit 26 (for 
example, a hard disc unit), a mouse 28 serving as a des- 
ignating device, and a keyboard 30. 
[01 62] An input/output (I/O) port 32, which is connect- 
ed to the camera input selector 14, supplies it with a 
camera control command. A video board 34 takes in the 
video signal from the video camera selected by the cam- 
era input selector 14, and displays various images on a 
bit-map display 35. A network interface 36 connects the 
video communication terminal apparatus 20 to a com- 
puter network or a communications network. The re- 
spective devices, namely, the CPU 22 to the video board 
34 and the network interface 36, are mutually connected 
by a system bus 38. The network interface 36 enables 
the video communication terminal apparatus 20 to con- 
trol a camera that is connected to a camera controller 
at a distant place by sending a control signal to the cam- 
era through the network, and to receive a camera control 
signal through the network. 

[01 63] The camera input selector 1 4 selects one con- 
trol signal line and its video output from control signal 
lines connected to the camera control circuits 12-1 to 
12-3 in order to supply the video board 34 with the se- 
lected video output, and logically connects the selected 
control line to the I/O port 32. As a video signal form, for 
example, the NTSC signal is used. The video board 34 
takes in a video signal outputted from the camera input 
selector 14. The video signal taken in is displayed as a 
moving-image in a predetermined window on the bit- 
map display 35. The video board 34 also has the func- 
tion of compressing or expanding the video signal. 
[0164] In the second memory device 26, a . variety of 
information related to the cameras 1 0-1 to 1 0-3 and oth- 
er cameras connected to the apparatus 20 through the 
network, for example, camera position information data, 
camera figure data and so forth are stored. 
[0165] When the camera input selector 14 is connec- 
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ed to only one of the video cameras 10-1 to 10-3, it does 
not need to be used. The corresponding camera con- 
troller 12 may be directly connected to the I/O port 32. 
When the video camera 10 is a type in which pan, tilt 
and so forth cannot be controlled, the camera control 
circuit 1 2 does not need to be used. When no video cam- 
era is used, in other words, when a video camera con- 
nected to another video communication terminal is op- 
erated by remote control to display only an image from 
the camera by using the video communication terminal 
apparatus 20, it is unnecessary to use any of the video 
cameras 10-1 to 10-3, any of the camera control circuits 
12-1 to 12-3 and the camera input selector 14. 
[0166] Fig. 30 shows an example of the video display 
window displayed on the bit-map display 35 when the 
video communication terminal apparatus 20 is connect- 
ed to four video cameras. In the bottom right corner of 
the display screen, buttons (camera selecting buttons) 
for selecting the cameras which output images to be dis- 
played within the display screen are displayed in a small 
size. The user can select one specified camera for dis- 
playing an image by selecting one button from these but- 
tons on the screen. A similar video display window is 
formed on the screen of another video communication 
terminal apparatus that receives an image from this vid- 
eo communication terminal apparatus and displays the 
received image. By operating camera selecting buttons 
on the screen at the receiving side, any of the video cam- 
eras connected to the transmitter's video communica- 
tion terminal can be selected and its output image can 
be displayed on the screen of the receiver's video com- 
munication terminal apparatus. 
[0167] As shown in Fig. 31 , the camera input selector 
14 may include a camera selector 14A for selecting a 
video camera to be controlled and supplying the select- 
ed video camera with a camera control signal, and a 
camera input composer 14B for outputting a composite 
video signal from all video cameras 1 0-1 to 1 0-3 or from 
a plurality of selected video cameras or for outputting a 
video signal from one selected camera. In such a case, 
selecting video cameras to be controlled, and selecting 
video cameras from which images are inputted to the 
camera input composer 14B are independent of each 
other. To produce the composite image, there are two 
techniques, spatial composition in which images from a 
plurality of video cameras are combined on the same 
screen, and time-based composition in which images 
from a plurality of video cameras are repeatedly 
changed over on a time-axis. In this embodiment both 
techniques can be selected. 

[0168] Figs. 32 and 33 show examples of the video 
display screen corresponding to the embodiment shown 
in Fig. 31. Fig. 32 shows a condition of the screen on 
which an image from a single video camera is displayed, 
while Fig. 33 shows a condition of the screen on which 
a composite image of four images from four video cam- 
eras is displayed. In each condition, camera selecting 
buttons and a composing button for switching over the 



composite image display and the single-image full- 
screen display are displayed in the bottom right corner 
of the screen. 

[0169] The video communication terminal apparatus 

5 shown in Fig. 29 or 31 is connected, as terminals 40, 42 
and 44, to a computer network 46, as shown in Fig. 34. 
The terminal 40, which is a monitoring terminal operated 
by the user as an observer, can control video cameras 
connected to itself and video cameras connected to the 

10 whole terminals 42, 44 connected to the network 46. The 
terminal 40 also can display an image from any camera 
or images from any cameras. Although one monitoring 
terminal 40 is used in general, there is no problem if a 
plurality of monitoring terminals 40 are used. 

15 [0170] Other terminals 42 and 44 are video transmis- 
sion terminals for transmitting an image or images from 
one camera (or from a plurality of cameras connected 
to the terminals) to the monitoring terminals 40. Al- 
though the terminal 42 has a structure as shown in Fig. 

20 29 and the terminal 44 has a structure as shown in Fig. 
31, it need hardly be said that the number of cameras 
connected to each terminal may be only one. In accord- 
ance with the number of cameras required, at least one 
video transmission terminal is used as the video trans- 

25 mission terminals 42 and 44. 

[0171] As the network 46, a local area network (LAN) 
or wide area network (WAN) that has a sufficient trans- 
mission bandwidth for transmitting both digital moving- 
image data and a camera control signal may be used. 

30 Such moving-image data may be transmitted with the 
data compressed. Since a variety of known methods 
can be utilized in this embodiment, a detailed descrip- 
tion thereof will be omitted. 

[01 72] Video boards 34 in the monitoring terminal 40, 

35 the video transmission terminals 42 and 44 (having the 
function of video capture) supply video data taken in to 
bit-map displays 35 to display images. A network inter- 
face 36 in the monitoring terminal 40 transmits control 
commands, such as a camera switching command and 

40 an image composing command, to the network 46, with 
the control commands converted into packets. These 
pieces of information are transmitted to a specified ter- 
minal in accordance with the contents of data to be 
transmitted and the necessity. The video transmission 

45 terminals 42 and 44 compress video data using their vid- 
eo boards 34 to convert it into packets, and transmit the 
packet-converted data to the monitoring terminal 40. 
[0173] On receiving the packet-converted video data 
from the video transmission terminal 42 or 44, the mon- 

50 itoring terminal 40 expands the received data using its 
video board 34 to display an image on the bit-map dis- 
play 35, similar to the processing of its local video signal. 
[0174] Fig. 35 is a block diagram showing the config- 
uration of software used in this embodiment. In the mon- 

55 itoring terminal 40, a camera managing server 50, a 
camera control server 52, a camera control client 54, 
and video reception software 56 are installed, while in 
the video transmission terminals 42 and 44, camera 
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control servers 52 and video transmission software 58 
are installed. 

[0175] The camera managing server 50 is software to 
manage all the cameras, connected to the network 46, 
which can be utilized by the monitoring terminal 40. The 
camera managing server 50 holds static information re- 
lated to the respective cameras, such as the names of 
the respective cameras, the names of hosts (the names 
of the terminals to which the cameras are connected), 
the locations at which the respective cameras are in- 
stalled, whether or not the panning, tilting and zooming 
of the cameras can be controlled, and dynamic informa- 
tion representing the present condition as to whether or 
not the cameras are being controlled or images from the 
cameras are being displayed, and the names of the ter- 
minals provided with the function of video composing 
(for example, the camera input composer 14B). The 
camera managing server 50 also manages the registra- 
tion of cameras which can be newly available through 
the network 46, the cancellation of cameras which has 
been separated from the network 46, and notifies the 
camera control client 54 of the management information 
of each camera, periodically or on demand. 
[0176] The camera control server 52 is software to 
control the cameras which belong thereto, being in- 
structed by the camera control client 54. The camera 
control server 52 displays the locations and directions 
where both the cameras connected to the monitoring 
terminal 40 and all the available cameras through the 
network 46 are installed so that predetermined corre- 
sponding camera symbols overlap with the map on the 
display screen of the monitoring terminal 40. The cam- 
era control client 54 also updates the form of the icon of 
each camera in real time, based on camera information 
sent from the camera managing server 50. 
[01 77] The video transmission software 58 compress- 
es an image taken in the terminal, and transmits the 
compressed image to a demanding terminal (normally, 
the monitoring terminal 40). The video reception soft- 
ware 56 expands the transmitted, compressed image, 
and displays it on the bit-map display. 
[0178] Fig. 36 shows a camera display control panel 
that is displayed on the bit-map display 35 of the moni- 
toring terminal 40 by the camera control client 54. A map 
window 60 displays camera icons which represent the 
locations and directions where the respective cameras 
are installed so that the icons overlap with a map show- 
ing the places where the cameras being controlled are 
installed. A camera display window 62 displays an im- 
age from one selected camera or images from a plurality 
of selected cameras (images from a maximum of six vid- 
eo transmitters), and includes six video display regions 
62a, 62b, 62c, 62d, 62e and 62f. A camera operation 
panel 64 is provided with various camera control buttons 
for operating the selected cameras to pan, tilt and zoom. 
On the bit-map display 35 of the monitoring terminal 40, 
a window display system that can simultaneously dis- 
play a plurality of windows operates. 



[0179] A map that shows the arrangement of seats in 
an office or store is displayed on the map window 60, 
and on this map the camera icons 66-1 to 66-12 which 
correspond to the respective locations of cameras in- 

5 stalled in such an office or store are displayed. The re- 
spective camera icons 66-1 to 66-12 are displayed at 
the appropriate positions corresponding to the locations 
of the cameras, almost in the same directions as the 
cameras are actually directed. The cameras corre- 

10 sponding to the camera icons 66-9 and 66-1 2 cannot be 
externally controlled, which condition is represented by 
the form of each icon shown in Fig. 37. The camera icon 
corresponding to a camera from which an image is being 
displayed on the video display window 62 is displayed 

15 by the camera control client 54 so that the frame of the 
icon is yellow. 

[0180] The operation of the camera display control 
panel shown in Fig. 36 and how the panel is used will 
be described below. By selecting one icon from the cam- 

20 era icons 66-1 to 66-12 on the map window 60 (for ex- 
ample, by clicking the icon 66-1 with a mouse), the cam- 
era corresponding to the selected icon is regarded as 
selected, and the frame color of the icon 66-1 is changed 
to yellowish green by the camera control client 54. The 

25 camera control client 54 queries the camera managing 
server 50 about information related to the camera cor- 
responding to the selected camera icon 66-1 . The cam- 
era managing server 50 makes reference to the name 
of the camera queried, and confirms whether or not an- 

30 other camera connected to the same host (video trans- 
mission terminal apparatus 20) is used. In other words, 
the camera managing server 50 looks up whether or not 
another camera connected to the camera input selector 
14 for the same host to which the selected camera is 

35 connected is used, and notifies the camera control client 
54 of the result. The camera managing server 50 simul- 
taneously confirms whether or not the host correspond- 
ing to the selected camera is provided with the camera 
input composer 1 4B. If the host is provided with the cam- 

40 era input composer 14B, the camera managing server 
50 notifies the camera control client 54 of its presence. 
[0181] The camera control client 54 changes each 
form of the camera icons corresponding to all the cam- 
eras connected to the same host to which the early se- 

45 lected camera is connected, into a form shown in Fig. 
38 when the host is not provided with the camera input 
composer 14B. The camera control client 54 changes 
each form into a form shown in Fig. 39 when the host is 
provided with the camera input composer 14B. The 

50 frames of the camera icons in both cases are changed 
to be yellowish green. While the image from the camera 
corresponding to the icon 66-1 is being displayed in one 
of the video display regions 62a to 62f of the video dis- 
play window 62, the camera control client 54 also chang- 

55 es the frame color of the video display region to yellow- 
ish green. 

[0182] For example, when the cameras correspond- 
ing to the camera icons 66-1, 66-2, 66-3 and 66-4 are 
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connected to the same host which is not provided with 
the camera input composer 14B, the camera control cli- 
ent 54 changes each form of the camera icons 66-1, 
66-2, 66-3 and 66-4 into the form shown in Fig. 38, which 
form has been already shown in Fig. 36, and displays 
the frames of the changed camera icons in yellowish 
green. While the image from the camera corresponding 
to the selected camera icon 66-1 is being displayed in 
the video display region 62a of the video display window 
62, the camera control client 54 displays the frame sur- 
rounding the video display region 62a in yellowish 
green, as shown in Fig. 36. 

[0183] Fig. 40 shows an example of the screen ob- 
tained when the camera corresponding to the selected 
camera icon (in this case the icon 66-1 ) is connected to 
the host provided with the camera input composer 14B. 
The camera control client 54 changes each form of the 
camera icons 66-1, 66-2, 66-3 and 66-4 into the form 
shown in Fig. 39, and displays the changed icon frame 
in yellowish green. As shown in Fig. 40, the composite 
image of images from four cameras connected to the 
host to which the camera corresponding to the camera 
icon 66-1 is connected is displayed in the video display 
region 62a of the video display window 62. Camera se- 
lecting buttons and composing buttons (as shown in 
Figs. 32 and 33) are displayed in the bottom right corner 
of the video display region 62a. In accordance with the 
selection of the camera icon 66-1, the camera control 
client 54 displays an L-shaped yellowish green frame 
along the periphery of a portion of the video display re- 
gion 62a, in which portion the image from the camera 
corresponding the camera icon 66-1 is displayed. An en- 
larged view of the portion of the video display region 62a 
is shown in Fig. 41. 

[0184] Such displayed camera icons on the map en- 
able the observer to easily grasp which cameras are 
connected to the same host to which the selected cam- 
era is connected. In addition, when the host is provided 
with the camera input composer, the display conditions 
of the camera icons are different, thus, the observer, can 
easily grasp both a type of cameras by which a compos- 
ite image of reduced images can be displayed and an- 
other type of camera by which a composite image of re- 
duced images cannot be displayed. 
[0185] In order to select a camera from which an im- 
age needs to be displayed on the video display window 
62, the camera icon corresponding to this camera may 
be dragged and dropped on the video display window 
62. For example, when the selected camera is repre- 
sented by the camera icon 66-2, the camera control cli- 
ent 54 queries the camera managing server 50 about 
whether or not an image from a camera connected to 
the same host to which the camera represented by the 
icon 66-2 is being displayed. 

[0186] If an image from a camera connected to the 
same host is being displayed, for example, while the im- 
age from the camera represented by the camera icon 
66-1 is being displayed, the camera control client 54 



sends the host a camera switching command for switch- 
ing the camera to the camera represented by the cam- 
era icon 66-2 so that the host transmits the image from 
the camera represented by the camera icon 66-2, and 
5 then displays the transmitted image from the camera 
represented by the camera icon 66-2 in the video display 
region (for example, 62a) where the image from the 
camera represented by the camera icon 66-1 is dis- 
played. 

10 [0187] While the image from the camera connected 
to the same host is not being displayed, the camera con- 
trol client 54 displays the image from the camera corre- 
sponding to the camera icon 66-2 in the display region 
where no image is displayed. At this time, the camera 

15 control client 54 looks up the host that is provided with 
the camera corresponding to the camera icon 66-2, and 
if another camera connected to the host is detected, 
camera selecting buttons are displayed in the video dis- 
play region in which the image from the camera corre- 

20 sponding to the camera icon 66-2 are displayed, as 
shown in Fig. 30. 

[0188] When an image is further displayed among the 
video display regions 62a to 62f of the video display win- 
dow 62, or when a camera from which an image is being 

25 displayed has been switched to another camera, the 
camera control managing server 50 updates the list of 
the names of the cameras from which images are being 
displayed, and the camera control client 54 changes the 
frames of camera icons corresponding to camera 

30 names included in the camera name list to be yellow. 
[0189] By selecting one image from the images dis- 
played in the video display regions 62a to 62f of the vid- 
eo display window 62 (for example, by clicking with the 
mouse), or by double clicking any camera icon on the 

35 map window 60, the camera operation panel 64 is dis- 
played. With respect to each of the video display regions 
62a to 62f for displaying an image from the camera cor- 
responding to the activated camera operation panel, a 
red frame surrounding the region is displayed to repre- 

40 sent a camera control condition. The color of the camera 
icon corresponding to the camera being controlled is al- 
so changed to red. 

[0190] In the foregoing embodiments the display 
forms of . the camera icons are changed in accordance 

45 with whether or not the cameras are controlled, whether 
or not the cameras are connected to the same host (vid- 
eo communication terminal apparatus 20), and whether 
or not the cameras are connected to the same camera 
input composer. However, for example, the brightness 

so of the camera icons corresponding to cameras which 
cannot be controlled may be changed such that the 
colors of the camera icons are darkened, or the display 
colors of the camera icons are differentiated. Similarly, 
although the camera icons corresponding to cameras 

55 from which images are displayed are discriminated 
based on their frame colors, the forms of the camera 
icons may be changed. The camera icons may be mod- 
ified by combining colors and forms so that the observer 
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can easily discriminate the icons. 
[0191] Further, it is obvious that the present invention 
may be applied to a system formed by a plurality of ap- 
paratuses or an apparatus formed by a single unit. In 
addition, the present invention may be applied by pro- 
viding a system or apparatus with software. In such a 
case a storage medium in which software for using the 
present invention is stored is a main component of the 
present invention. The system or apparatus operates in 
a predetermined manner by the software read from the 
storage medium thereto. 

[0192] As easily understood from the foregoing de- 
scription, in accordance with the above embodiments, 
the camera icons displayed on the map enable the ob- 
server to easily grasp the relationship between the cam- 
eras represented by the camera icons and the images 
being displayed on the video display screen. 
[0193] In addition, the cameras which can be exter- 
nally controlled to pan, tilt and zoom, and the cameras 
which cannot be externally controlled are discriminated 
by using the camera icons. 

[0194] Further, the camera icons on the map enable 
the observer to recognize which camera is connected 
to the same video transmission apparatus to which the 
selected camera is connected, and to recognize which 
cameras are connected to the same camera input com- 
poser. Accordingly, the operability of the system is ex- 
tremely improved. 

[0195] As described above, in accordance with the 
above embodiments, camera information can be dis- 
played by using the camera icons on the map so that 
the observer can easily understand the information. 



Claims 

1. A monitor control apparatus for monitoring the op- 
eration of a plurality of remote cameras, the appa- 
ratus comprising: 

video generating means (136,413) for supply- 
ing a video signal to a monitor means so as to 
display a first screen area (500), which screen 
area allows a user to select one of a plurality of 
locations (510,520 ...) using a pointer so that a 
map of the selected location is displayed in the 
first screen area, the map indicating a plurality 
of symbols each representing a camera, 

and characterised in that when an operator 
selects a location said video generating means 
causes a display in a separate screen area which 
has a plurality of separate windows (610,620...) 
and enables the operator to successively select 
from a plurality of camera symbols by dragging and 
dropping each camera symbol into selected ones 
of said windows whereby the video outputs of the 
selected cameras are displayed in the respective 



windows in which the camera symbols have been 
dropped. 

2. Apparatus according to claim 1 , comprising control 
5 means (41 0,41 1 ) for establishing a network for im- 
age communication with said cameras in accord- 
ance with said drag and drop. 

3. Apparatus according to claim 2, wherein said net- 
10 work is a local area network. 

4. Apparatus according to claim 3, wherein said net- 
work is a bidirectional communication network. 

15 5. Apparatus according to either of claims 2 or 3, 
wherein said network is a packet communication 
network. 

6. Apparatus according to any preceding claim, further 
20 comprising generating means for generating infor- 
mation indicating a photographing condition of said 
cameras. 

7. Apparatus according to claim 6, wherein said video 
25 generating means is adapted to cause display of, 

said photographing condition on said monitor. 

8. Apparatus according to claim 6 or claim 7, wherein 
said photographing condition comprises at least 

30 one of a photographing direction, a photographing 
view angle, and an in-focus position of said image 
signal generating means. 

9. Apparatus according to any one of the preceding 
35 claims in combination with a plurality of video cam- 
eras. 

10. A method of monitoring the operation of a plurality 
of remote cameras, the method comprising: 

40 

supplying a video signal to a monitor means so 
as to display a first screen area (500), which 
screen area allows a user to select one of a plu- 
rality of locations (510,520 ...), using a pointer 
45 so that a map of the selected location is dis- 

played in the first screen area, the map indicat- 
ing a plurality of symbols each representing a 
camera, 

so and characterised in that when an operator 

selects a location a display is caused in a separate 
screen area which has a plurality of separate win- 
dows (610,620 ...), the operator successively se- 
lecting from a plurality of camera symbols by drag- 

55 ging and dropping each camera symbol into select- 
ed ones of said windows whereby the video outputs 
of the selected cameras are displayed in the re- 
spective windows in which the camera symbols 
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have been dropped. 

11. A method according to claim 10, further including 
generating information indicating photographing 
conditions of said cameras. 5 



Patentanspriiche 

1. Oberwachungssteuergerat zur Oberwachung der 10 
Operation einer Vielzahl von Fernkameras, mit 

einerVideoerzeugungseinrichtung (136, 413) 
zum Zufuhren eines Videosignals zu einer Oberwa- 
chungseinrichtung, urn so einen ersten Bildschirm- 
bereich (500), welcher einem Benutzer ermoglicht, 15 
einen einer Vielzahl von Orten (510, 520 ...) auszu- 
wahlen, unter Verwendung eines Zeigers anzuzei- 
gen, so dass eine Karte des ausgewahlten Orts in 
dem ersten Bildschirmbereich angezeigt wird, wo- 
bei die Karte eine Vielzahl von Symbolen zu erken- 20 
nen gibt, welches jeweils eine Kamera reprasen- 
tiert, 

und dadurch gekennzeichnet, dass wenn 
eine Bedienperson einen Ort auswahlt, die Video- 
erzeugungseinrichtung eine Anzeige in einem eine 25 
Vielzahl von separaten Fenstern (610, 620 ...) auf- 
weisenden separaten Bildschirmbereich bewirkt, 
und die Bedienperson in die Lage versetzt, nach- 
einander aus einer Vielzahl von Kamerasymbolen 
durch Ziehen und Ablegen jedes Kamerasymbols 30 
in ausgewahlte Fenster der Fenster auszuwahlen, 
wodurch die Videoausgaben der ausgewahlten Ka- 
meras in den je^feiligen Fenstern angezeigt wer- 
den, in welchen die Kamerasymbole abgelegt wur- 
den. 35 

2. Gerat nach Anspruch 1 , mit einer Steuereinrichtung 
(410, 411) zum Errichten eines Netzwerks zur Bild- 
kommunikation mit den Kameras gemafc dem Zie- 
hen und Ablegen. *o 

3. Gerat nach Anspruch 2, wobei das Netzwerk ein lo- 
kales Netzwerk ist. 

4. Gerat nach Anspruch 3, wobei das Netzwerk ein bi- *s 
direktionales Kommunikationsnetzwerk ist. 

5. Gerat nach entweder Anspruch 2 oder 3, wobei das 
Netzwerk ein Paketkommunikationsnetzwerk ist. 

50 

6. Gerat nach einem der vorangehenden Anspruche, 
zudem mit einer Erzeugungseinrichtung zur Erzeu- 
gung von Informationen, die einen Photographier- 
zustand der Kameras zu erkennen geben. 

55 

7. Gerat nach Anspruch 6, wobei die Videoerzeu- 
gungseinrichtung dahingehend ausgestaltet ist, urn 
eine Anzeige des Photographierzustands auf der 



Oberwachungseinrichtung zu bewirken. 

8. Gerat nach Anspruch 6 oder Anspruch 7, wobei der 
Photographierzustand zumindest einen Zustand ei- 
ner Photographierrichtung, einem Photographier- 
sichtwinkel, und einer im Blickfeld gelegenen Posi- 
tion der Bildsignal-Erzeugungseinrichtung auf- 
weist. 

9. Gerat nach einem der vorangehenden Anspruche 
in Kombination mit einer Vielzahl von Videokame- 
ras. 

10. Verfahren zum Oberwachen der Operation einer 
Vielzahl von Fernkameras, mit 

Zufuhren eines Videosignals zu einer Ober- 
wachungseinrichtung, urn so einen ersten Bild- 
schirmbereich (500), welcher einem Benutzer er- 
moglicht, einen einer Vielzahl yon Orten (510, 
520 ...) auszuwahlen, unter Verwendung eines Zei- 
gers anzuzeigen, so dass eine Karte des ausge- 
wahlten Orts in dem ersten Bildschirmbereich an- 
gezeigt wird, wobei die Karte eine Vielzahl von 
Symbolen zu erkennen gibt, welches jeweils eine 
Kamera darstellt, 

und dadurch gekennzeichnet, dass wenn 
eine Bedienperson einen Ort auswahlt, in einem ei- 
ne Vielzahl von separaten Fenstern (610, 620 ...) 
aufweisenden separaten Bildschirmbereich eine 
Anzeige bewirkt wird, wobei die Bedienperson 
nacheinander aus einer Vielzahl von Kamerasym- 
bolen durch Ziehen und Ablegen jedes Kamera- 
symbols in ausgewahlte Fenster der Fenster aus- 
wahlt, wodurch die Videoausgaben der ausgewahl- 
ten Kameras in den jeweiligen Fenstern angezeigt 
werden, in welchen die Kamerasymbole abgelegt 
wurden. 

11. Verfahren nach Anspruch 10, zudem mit Erzeugen 
von Informationen, die Photographierzustande der 
Kameras zu erkennen geben. 



Revendications 

1. Appareil de commande d'un moniteur pour le con- 
trole du fonctionnement de plusjeurs cameras eloi- 
gn6es, I'appareil comportant : 

un moyen de g§n6ration video (136, 413) des- 
tine a fournir un signal vid6o a un moyen a mo- 
niteur afin d'afficher une premiere zone d'ecran 
(500), laquelle zone d'6cran permet a un utili- 
sateur de selectionner Tun de plusieurs empla- 
cements (51 0, 520) en utilisant un pointeur afin 
d'afficher une carte de ('emplacement s6lec- 
tionne dans la premiere zone d'6cran, la carte 
indiquant plusieurs symboles repr^sentant 
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chacun une camera, 

et caracterise en ce que, lorsqu'un opgrateur 
selectionne un emplacement, ledit moyen de gene- 
ration video provoque un affichage dans une zone 
d'ecran s6par6e qui comporte plusieurs fenetres 
separees (610, 620 ...) et permet a I'operateur de 
selectionner successivement parmi plusieurs sym- 
boles de cameras en glissant et deplagant chaque 
symbole de camera jusque dans certaines, selec- 
tionnees, desdites fenetres, grace a quoi les si- 
gnaux de sortie video des cameras selectionnees 
sont affiches dans les fenetres respectives dans 
lesquelles les symboles de cameras ont 6te places. 

2. Appareil selon la revendication 1, comportant un 
moyen de commande (410, 411) destine a etablir 
un reseau pour une communication d'images avec 
lesdites cameras conformement audit glisse/depla- 
ce. 

3. Appareil selpn la revendication 2, dans lequel ledit 
reseau est un reseau local. 

4. Appareil selon la revendication 3, dans lequel ledit 
reseau est un reseau de communications bidirec- 
tionnelles. 



niteur afin d'afficher une premiere zone d'ecran 
(500), laquelle zone d'ecran permet a un utili- 
sateur de selectionner Tun de plusieurs empla- 
cements (510, 520 ...), en utilisant un pointeur 
5 afin q'une carte de I'emplacement selectionne 

soit affichee dans la premiere zone d'ecran, la 
carte indiquant plusieurs symboles represen- 
tant chacun une camera, 

10 et caracterise en ce que, lorsqu'un operateur 

selectionne un emplacement, un affichage est en- 
gendre dans une zone d'ecran separee qui compor- 
te plusieurs fenetres separees (610, 620 ...), I'ope- 
rateur selectionnant successivement parmi plu- 

15 sieurs symboles de cameras en amenant par glis- 
se/d6place chaque symbole de camera dans cer- 
taines, selectionnees, desdites fenetres grace a 
quoi les signaux de sortie video des cameras selec- 
tionnees sont affiches dans les fenetres respectives 

20 dans lesquelles les symboles de cameras ont ete 
places. 

11. Procede selon la revendication 10, comprenant en 
outre la generation d'une formation indiquant des 
25 conditions de prise de vue desdites cameras. 



5. Appareil selon Tune des revendications 2 ou 3, dans 
lequel ledit reseau est un reseau de communica- 30 
tions par paquets. 

6. Appareil selon I'une quelconque des revendications 
precedentes, comportant en outre un moyen de ge- 
neration destine a generer une information indi- 35 
quant une condition de prises de vue desdites ca- 
meras. 



Appareil selon la revendication 6, dans lequel ledit 
moyen de generation video est congu pour provo- *o 
quer un affichage de ladite condition de prise de vue 
sur ledit moniteur. 

Appareil selon la revendication 6 ou la revendica- 
tion 7, dans lequel ladite condition de prise de vue *s 
comprend au moins Tun d'une direction de prise de 
vue, d'un angle de visee de prise de vue et d'une 
position de mise au point dudit moyen de genera- 
tion de signal d'image. 



Appareil selon Tune quelconque des revendications 
precedentes en combinaison avec plusieurs came- 
ras video. 



50 



10. Procede de controle du fonctionnement de plu- 55 
sieurs cameras eioignees, le procede comprenant : 



la fourniture d'un signal video & un moyen a mo- 
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